Patient monitor

ABSTRACT

Images obtained by a camera system ( 10 ) arranged to obtain images of a patient ( 20 ) undergoing radio-therapy are processed by a modeling unit ( 56,58 ) which generates a model of the surface of a patient ( 20 ) being monitored. Additionally the patient monitoring system processes image data not utilized to generate a model of the surface of a patient being monitored to determine further information concerning the treatment of the patient ( 20 ). Such additional data can comprise data identifying the relative location of the patient and a treatment apparatus ( 16 ). This can be facilitated by providing a number or retro-reflective markers ( 30 - 40 ) on a treatment apparatus ( 16 ) and a mechanical couch ( 18 ) used to position the patient ( 20 ) relative to the treatment apparatus ( 16 ) and monitoring the presence and location of the markers in the portions of the images obtained by the stereoscopic camera ( 10 ).

This application is a Continuation of co-pending application Ser. No.16/230,739 filed on Dec. 21, 2018, which is a Continuation ofapplication Ser. No. 14/424,376 filed on Feb. 26, 2015 (now U.S. Pat.No. 10,183,177 issued on Jan. 22, 2019), which is a National Phase ofPCT International Application No. PCT/GB2013/052650 filed on Oct. 11,2013, which claims priority under 35 U.S.C. § 119(a) to PatentApplication No. 1218307.5 filed in the United Kingdom on Oct. 12, 2012.All of the above applications are hereby expressly incorporated byreference into the present application.

The present invention relates to patient monitoring. More particularly,embodiments of the present invention relate to monitoring thepositioning of patients and also to enable the movement of patients tobe detected. The invention is particularly suitable for use withradiotherapy devices and the like where accurate positioning and thedetection of patient movement is important for successful treatment.

Radiotherapy consists of projecting onto a predetermined region of apatient's body, a radiation beam so as to destroy or eliminate tumoursexisting therein. Such treatment is usually carried out periodically andrepeatedly. At each medical intervention, the radiation source must bepositioned with respect to the patient in order to irradiate theselected region with the highest possible accuracy to avoid radiatingadjacent tissue on which radiation beams would be harmful.

When applying radiation to a patient, the gating of treatment apparatusshould be matched with the breathing cycle so that radiation is focusedon the location of a tumour and collateral damage to other tissues isminimized. If movement of a patient is detected the treatment should behalted to avoid irradiating areas of a patient other than a tumourlocation.

For this reason a number of monitoring systems for assisting thepositioning of patients during radiotherapy have therefore been proposedsuch as those described in Vision RT's earlier patents and patentapplications U.S. Pat. Nos. 7,348,974, 7,889,906 and US2009-018711 allof which are hereby incorporated by reference.

In the systems described in Vision RT's patent applications,stereoscopic images of a patient are obtained and processed to generatedata identifying 30 positions of a large number of points correspondingto points on the surface of an imaged patient. Such data can be comparedwith data generated on a previous occasion and used to position apatient in a consistent manner or provide a warning when a patient movesout of position. Typically such a comparison involves undertakingProcrustes analysis to determine a transformation which minimizes thedifferences in position between points on the surface of a patientidentified by data generated based on live images and points on thesurface of a patient identified by data generated on a previousoccasion.

Treatment plans for the application of radiotherapy are becomingincreasingly complex with treatment apparatus having multiple orfloating iso-centres. Such increasing complexity brings with itincreasing possibilities of mistreatment. There is therefore a need fora monitoring system which can detect when errors occur and halttreatment when such errors are detected.

SUMMARY OF INVENTION

In accordance with one aspect of the present invention there is provideda patient monitoring system for monitoring a patient undergoingradiotherapy comprising: a stereoscopic camera system operable to obtainstereoscopic images of a patient undergoing radiotherapy; a modelingunit operable to process stereoscopic images of a patient and generate amodel of the surface of a patient being monitored on the basis of theappearance of the patient corresponding to part of the stereoscopicimages obtained by the stereoscopic camera system; wherein the patientmonitoring system is arranged to process image data corresponding toportions of the stereoscopic images which are not utilized to generate amodel of the surface of a patient being monitored to determine furtherinformation concerning the treatment of the patient.

The patient monitoring system may comprise an apparatus positiondetermination module operable to process stereoscopic images obtained bythe stereoscopic camera system and utilize portions of the stereoscopicimages which are not utilized to generate a model of the surface of apatient being monitored to determine the relative positioning of atreatment apparatus relative to a patient.

The patient monitoring system may be incorporated in a treatment systemwhich includes a treatment apparatus for treating a patient; and amechanical couch for positioning a patient relative to the treatmentapparatus. In such a system a plurality of markers may be attached tothe surface of the treatment apparatus and the patient monitoring systemmay be arranged to process images of the markers contained in portionsof the stereoscopic images which are not utilized to generate a model ofthe surface of a patient to determine the relative positioning of atreatment apparatus relative to a patient.

The treatment apparatus may comprise a gantry operable to rotate aboutan axis relative to a main body of the apparatus. In such a system, atleast some of the markers may be provided on the surface of thetreatment apparatus arranged such that the apparatus positiondetermination module is operable to determine the relative rotation ofthe gantry about the axis on the basis of the positions of the markersin the portions of the stereoscopic images which are not utilized togenerate a model of the surface of a patient.

In some systems the treatment apparatus may comprise a rotatablecollimator provided at the end of the gantry remote from the main bodyof the apparatus where the rotatable collimator is operable to rotateabout an axis perpendicular to the axial extent of the gantry. In such asystem, at least some of the markers may be provided on the surface ofthe treatment apparatus arranged such that the apparatus positiondetermination module is operable to determine the relative rotation ofthe collimator on the basis of the positions of the markers in theportions of the stereoscopic images which are not utilized to generate amodel of the surface of a patient. In some embodiments this may beachieved by the markers being provided on the surface of the treatmentapparatus in a number of groups and the apparatus position determinationmodule may be operable to determine the extent of rotation based on thepresence of one or more groups of markers in the portions of thestereoscopic images which are not utilized to generate a model of thesurface of a patient.

In some embodiments one or more markers may be attached to themechanical couch and the apparatus position determination module may beoperable to determine the relative positions of the mechanical couchrelative to the position of the treatment apparatus on the basis of theposition of the one or more markers attached to the mechanical couch inthe portions of the stereoscopic images which are not utilized togenerate a model of the surface of a patient.

Where markers are attached to the treatment apparatus or the mechanicalcouch, the markers may comprise retro-reflective markers and thestereoscopic camera system may include a light source for illuminatingthe markers. In such an embodiment the apparatus position determinationmodule may be arranged to identify portions of the stereoscopic imageswhich are not utilized to generate a model of the surface of the patientand which correspond to the markers by performing a thresholdingoperation.

In some embodiments, the markers utilized may comprise spherical markersand the apparatus position determination module may be arranged toidentify portions of the stereoscopic images which are not utilized togenerate a model of the surface of the patient and which correspond tothe markers by identifying circular representations of the markers inthe images.

The stereoscopic camera system may be operable to obtain a sequence ofimages of a patient undergoing radiotherapy lying on the mechanicalcouch. In such a system the apparatus position determination module maybe arranged to utilize the detected positions of the markers in thesequence of images to track the relative positioning of a treatmentapparatus relative to a patient.

Having determined the relative positioning of a treatment apparatusrelative to a patient the monitoring system may then compare therelative positioning of a treatment apparatus relative to a patient withan expected relative positioning based on a predefined treatment planand provide a warning or halt treatment if the detected positioning ofthe treatment apparatus does not match with that expected on the basisof the treatment plan.

In addition to confirming that the positioning of the treatmentapparatus and the patient correspond with a treatment plan the systemmay additionally determine the relative positioning of a treatmentapparatus relative to a patient and generate a warning if the treatmentapparatus is likely to collide with the patient or the mechanical couch.

In some embodiments the positioning of a patient may be facilitated bythe use of a face mask worn by a patient and attached to the mechanicalcouch. In such embodiments the use of the correct face mask may bedetermined by providing a face mask having distinctive markings thereon.Subsequently the patient monitoring system may process image datacorresponding to portions of the stereoscopic images which are notutilized to generate a model of the surface of a patient to detect thepresence of the distinctive markings on a face mask and provide awarning if the markings do not correspond to the expected markings for apatient being treated.

In addition to facilitating the checking and confirmation of theadherence to a defined treatment plan the provision of a treatmentapparatus and mechanical couch having markers attached also facilitatesthe calibration of a stereoscopic camera system. In particularmonitoring the location of markers attached to a treatment apparatusenables the plane of rotation of a treatment apparatus to be establishedand hence establish an appropriate co-ordinate system for modeling thelocation of the patient relative to the axes of movement of thetreatment apparatus. Additionally monitoring the motion of markers on amechanical couch can enable co-ordinates for a volume of interest formonitoring a patient to be mapped out.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will now be described withreference to the accompanying drawings in which:

FIG. 1 is a schematic perspective view of a patient monitor inaccordance with an embodiment of the present invention;

FIG. 2 is a front perspective view of the camera system of the patientmonitor of FIG. 1;

FIG. 3 is a schematic block diagram of the computer system of thepatient monitor of FIG. 1;

FIG. 4 is a flow diagram of the processing undertaken by the apparatusposition determination module in the computer system of the patientmonitor of FIG. 1; and

FIGS. 5A and 5B are illustrative representations of images captured bythe camera system of the patient monitor of FIG. 1.

DETAILED DESCRIPTION

FIG. 1 is a schematic perspective view of a patient monitor inaccordance with an embodiment of the present invention. In accordancewith this embodiment, there is provided set of stereoscopic cameras 10that are connected by wiring (not shown) to a computer 14. The computer14 is also connected to treatment apparatus 16 such as a linearaccelerator for applying radiotherapy. A mechanical couch 18 is providedas part of the treatment apparatus upon which a patient 20 lies duringtreatment. The treatment apparatus 16 and the mechanical couch 18 arearranged such that, under the control of the computer 14, the relativepositions of the mechanical couch 18 and the treatment apparatus 16 maybe varied, laterally, vertically, longitudinally and rotationally as isindicated in the figure by the arrows adjacent the couch.

The treatment apparatus 16 comprises a main body 22 from which extends agantry 24. A collimator 26 is provided at the end of the gantry 24remote from the main body 22 of the treatment apparatus 16. To vary theangles at which radiation irradiates a patient 20, the gantry 24, underthe control of the computer 14, is arranged to rotate about an axispassing through the centre of the main body 22 of the treatmentapparatus 16. Additionally the location of irradiation by the treatmentapparatus may also be varied by rotating the collimator 26 at the end ofthe gantry 24.

In use, the stereoscopic cameras 10 obtain video images of a patient 20lying on the mechanical couch 18. These video images are passed via thewiring to the computer 14. The computer 14 then processes the images ofthe patient 20 to generate a model of the surface of the patient. Thismodel is compared with a model of the patient generated during earliertreatment sessions. When positioning a patient the difference between acurrent model surface and a target model surface obtained from anearlier session is identified and the positioning instructions necessaryto align the surfaces determined and sent to the mechanical couch 18.Subsequently during treatment any deviation from an initial set up canbe identified and if the deviation is greater than a threshold, thecomputer 14 sends instructions to the treatment apparatus 16 to causetreatment to be halted until a patient 20 can be repositioned.

The applicants have appreciated that the surface of a patient 20 whichcan be monitored using a set of stereoscopic camera system 10 accountsfor only a relatively small portion of the camera system's field ofview. Thus for example when monitoring a patient only some or all of apatient's upper torso might be monitored as matching and monitoring suchan area is sufficient for the purposes of patient positioning and fordetecting patient movement or irregular breathing or the like. Indeedlimiting the processing to generate surface data representing only alimited surface area is preferable as limiting the surface area to bemonitored reduces the amount of processing necessary to convert imagedata into a surface model.

Further the applicants have also appreciated that as not all of theimage data captured by a stereoscopic camera is required to monitorpatient positioning, other portions of the image data can be used forother purposes.

More specifically, as is illustrated in FIG. 1 by dashed lines portionsof the treatment apparatus 16 and the mechanical couch 18 are in thefield of view of the camera system 10 in addition to portions of thepatient 20 which is being monitored. Based on that appreciation, it hasbecome apparent to the applicants that image data captured by thestereoscopic camera system 10 could be utilized to capture real-timeinformation about the positioning of the treatment apparatus 16 and themechanical couch 18 during treatment. Such data capture could then beutilized to confirm that the treatment apparatus 16 and the mechanicalcouch 18 were being positioned relative to the patient 20 and hencecausing the patient 20 to be irradiated with radiation in accordancewith a programmed irradiation program.

As will be described in greater detail, the monitoring of the relativepositions of the treatment apparatus 16 and a patient 20 can befacilitated by providing a number of markers 30-40 attached to thesurface of the main body of the treatment apparatus, the gantry 24, thecollimator 26 and the couch 18 respectively. By providing thestereoscopic camera system 10 with an appropriate light source andmaking the markers reflective, it becomes possible to identify theportions of the images corresponding to the markers 30-40 by performinga thresholding operation. This enables the portions of imagescorresponding to the markers 30-40 to be rapidly identified. Theprocessing of images to identify portions of images corresponding to themarkers 30-40 can additionally be facilitated by using the expected ormonitored positions of the treatment apparatus 16 and mechanical couch18 to identify the expected portions of an image where markers 30-40 areexpected to be viewed.

The positioning and format of the markers 30-40 need to be chosen toaccount for different portions of the treatment apparatus 16 and couch18 being obscured at different times depending on their relativelocations and the position of the patient 20 on the mechanical couch 18.Additionally where reflective markers are utilized, the relativepositioning of the stereoscopic camera 10 and the markers 30-40 needs tobe taken into account.

One suitable arrangement of markers 30-40 is to place the markers 30-40as follows.

In order to track the rotation of the gantry 24 relative to the mainbody of the treatment apparatus, two markers 30,31 (marker 31 is notvisible in FIG. 1, but is located on the remote side of the gantry 24,in a correspondingly symmetrical position to marker 30) may be placed onthe gantry head and two markers 32,33 may be placed on the body portionof the treatment apparatus adjacent the main body 22 of the apparatus16, the markers 30-33 thereby identifying the four corners of a squareor rectangle. This arrangement of four markers 30-33 ensures that at alltimes 3 or more markers are in the field of view of a stereoscopiccamera system 10 placed opposite the treatment apparatus 16 such as isillustrated in FIG. 1. In the arrangement such as is illustrated in FIG.1 where the camera planes of the cameras of the stereoscopic camerasystem 10 are substantially aligned with the plane of rotation of thegantry 24, the markers 30-33 used to track the rotation of the gantry 24can be portions of reflective material stuck on to the surface of thetreatment apparatus viewed by the camera system 10. The alignment ofcamera planes of the cameras of the stereoscopic camera system 10 andthe plane of rotation of the gantry 24 simplifies tracking as rotationof the gantry 24 should only cause the apparent positions of the markers30-33 to vary and not change the apparent size or shape of the markers30-33.

In order to maximize the accuracy with which the rotation of the gantry24 can be monitored, it is preferred that the shape and appearance ofthe markers 30-33 should be such that the centres of the markers 30-33as they appear in images obtained by the stereoscopic camera system 10can be easily determined. In this embodiment this is achieved byutilizing circular markers and processing image data to identify thecentres of the imaged markers 30-33.

In the case of the collimator 26, where the camera planes of the camerasof the stereoscopic camera system 10 are substantially aligned with theplane of rotation of the gantry 24, the collimator 26 will be arrangedto rotate in a plane which is at right angles to the camera planes ofthe cameras of the stereoscopic camera system 10. This will mean that asthe collimator 26 rotates different portions of the collimator 26 willbe presented in the field of view of the stereoscopic camera system 10.Because of this in order to track the rotation of the collimator 26, itis preferred that the markers 34-38 present on the collimator 26 shouldbe grouped in distinctive patterns so that the presence or absence ofparticular patterns or markers 34-38 can enable the rotational positionof the collimator 26 to be inferred. Thus for example the markers 34-38might be arranged in four groups spaced equally around the circumferenceof the collimator 26 in slightly different patterns e.g. one group mightconsist of a single marker 38 (not shown in FIG. 1 but visible in FIGS.5A and 58), one group might consist of two markers 36, 37 immediatelyadjacent each other, another group might consist of two markers 34, 35spaced apart and a third group might consist of three markers (not shownin the Figures) on the opposite side of the collimator 26 to one of theother groups. In the illustrated embodiment, this would be the side ofthe collimator 26 opposite and remote from the pair of spaced apartmarkers 34, 35. In this way whenever two groups of markers were visible,the identity of the groups could be identified and the rotationalposition of the collimator inferred from the apparent position of thevisible markers and the identity of the visible groups.

As with the markers 30-33 for tracking the rotation of the gantry 24 itis preferable that the shape an appearance of the markers 34-38 shouldbe such that the centres of the markers 34-38 for tracking the rotationof the collimator 26 should also be easily determined. As the rotationof the collimator 26 varies the location, distance and orientation ofmarkers 34-38 on the collimator 26 relative to the camera planes of thecameras of the stereoscopic camera system 10, as well as the shape andappearance of the markers 34-38 will vary as the collimator 26 isrotated. This variance can be reduced by utilizing spherical markers asonly the apparent size and not the apparent shape of the markers willvary, hence simplifying the identification of the locations of centre ofthe markers.

The location and orientation of the mechanical couch 18 can be trackedusing a pair of markers 39,40 located near or adjacent the portion ofthe patient 20 which is imaged by the stereoscopic camera system 10. Ina typical treatment system normally there is capacity for attachingadditional equipment to a mechanical couch 18 for treatment purposes andsuitable tracking markers 39,40 can be attached in the same way.Suitable locations can include being placed on the surface of the bed ofthe mechanical couch 18 adjacent where the head of a patient will lie 40or alternatively markers may be attached to the edge 39 of the bed ofthe couch in the vicinity of the portion of the patient 20 being imaged.

Again as in the case of markers for tracking the rotation of thecollimator 26, it is preferable to utilize spherical markers fortracking the location and orientation of the mechanical couch 18 as therelative orientation and distance of the markers 39,40 and the cameraplane of the cameras in the stereoscopic camera system 10 will vary asthe location and orientation of the couch 18 is varied.

Monitoring the location of a patient 20 and the orientation and movementof a treatment apparatus 16 and a mechanical couch 18 utilizing the samestereoscopic camera 10 gives rise to a number of advantages.

Firstly monitoring both the position of the patient 20 and the treatmentapparatus 16 during treatment enables the application of radiation to becompared in real time with a treatment plan. The timing and movement ofa treatment apparatus 16 in accordance with a treatment plan is usuallydetermined relative to the iso-centre of the treatment apparatus (i.e. afixed point irradiated by the treatment apparatus 16 and about which thetreatment apparatus 16 is considered to rotate. In theory the mechanicalcouch 18 causes a tumour in the patient 20 to be positioned at thatiso-centre and the treatment causes the tumour to be irradiated fromdifferent angles in the course of treatment. It is, however, ultimatelythe irradiation of the tumour rather than irradiation of the iso-centrewhich is ultimately important and by monitoring the surface of thepatient 20 and the movement of the treatment apparatus 16 it is possiblefor the actual irradiation of a tumour site to be monitored.

In addition to being able to confirm the actual history of irradiationof a tumour site in terms of a position relative to a detected surfaceof a patient 20, the described system also facilitates improved qualitycontrol and confirmation of the accuracy of the stereoscopic camerasystem 10. Thus for example, by arranging for markers 39,40 to beattached to a mechanical couch 18 when calibrating the system, it ispossible to move the mechanical couch 18 and hence the attached markers39,40 predefined distances in predefined directions. This enables thecalibration of the stereoscopic camera system 10 to be performed acrossa wide volume which is liable to subsequently contain the surface of apatient 20 being monitored and hence the stereoscopic camera system 10can be calibrated accurately throughout this volume of interest. Whencalibrating the stereoscopic camera 10 it is also desirable for theco-ordinate system of the stereoscopic camera 10 to match that used bythe treatment plan for the treatment apparatus 16 and mechanical couch18. One way this can be done is through monitoring a calibration modelof known dimensions on the surface of the mechanical couch 18 and thenmoving the couch laterally, horizontally and vertically to determineaxes corresponding to the axes of couch motion. Monitoring the movementof the treatment apparatus 16 itself provides further confirmation ofthe co-ordinate system the treatment apparatus is arranged to operate.In contrast to the mechanical couch 18, a treatment apparatus 16 isnormally only arranged to rotate about a fixed point. This means thatmonitoring the position of markers attached to the treatment apparatusshould identify a plane of rotation containing the markers 30-33 whichin turn should correspond with the co-ordinate system for couchmovement. This additional information can again be utilized to calibratethe co-ordinate system for the stereoscopic cameras 10 and any change inthe measured plane of movement of the markers can be detected andinvestigated. Again as with the monitoring of motion of the mechanicalcouch 18 such calibration and confirmation can be determined for arelatively large volume and hence the reliability of calibration can beincreased.

FIG. 2 is a front perspective view of the camera system 10 of thepatient monitor of FIG. 1.

In this embodiment the camera system 10 comprises a housing 41 which isconnected to a bracket 42 via a hinge 44. The bracket 42 enables thecamera system 10 to be attached in a fixed location to the ceiling of atreatment room whilst the hinge 44 permits the orientation of the camerasystem 10 to be orientated relative to the bracket 42 so that the camerasystem 10 is arranged to view a patient 20 on a mechanical couch 18.

A pair of lenses 46 is mounted at either end of the front surface 48 ofthe housing 41. These lenses 46 are positioned in front of imagedetectors such as CMOS active pixel sensors or charge coupled devices(not shown) contained within the housing 41. The image detectors arearranged behind the lenses 46 so as to capture images of a patient 20via the lenses 46.

A set of LED lights 50 is positioned around the outside of thecircumference of each of the lenses 46. The LED lights 50 are orientatedto illuminate the field of view of the camera system 10 and inparticular the retro-reflective markers 30-40 attached to the treatmentapparatus 16 and the mechanical couch 18.

A speckle projector 52 is provided in the middle of the front surface 48of the housing 41 between the two lenses 46. The speckle projector 52 isarranged to illuminate a patient 20 with a non-repeating speckledpattern of infrared light so that when images of a patient 20 arecaptured by the two image detectors corresponding portions of capturedimages can be distinguished. To that end the speckle projector comprisesa light source such as a LED and a film with a random speckle patternprinted on the film. In use light from the light source is projected viathe film and as a result a pattern consisting of light and dark areas isprojected onto the surface of a patient 20. When images of the projectedspeckle pattern are captured by the camera system 10 the images can thenbe processed to determine the positions of a set of points on thesurface of the patient and hence the positioning of the patient can bemonitored.

FIG. 3 is a schematic block diagram of the computer 14 of the patientmonitor of FIG. 1.

In order for the computer 14 to process images received from thestereoscopic cameras 10, the computer 14 is configured by softwareeither provided on a disk 54 or by receiving an electrical signal 55 viaa communications network into a number of functional modules 56-66. Itwill be appreciated that the functional modules 56-66 illustrated inFIG. 3 are purely notional in order to assist with the understanding ofthe working of the claimed invention and may not in certain embodimentsdirectly correspond with blocks of code in the source code for thesoftware. In other embodiments the functions performed by theillustrated functional modules 56-66 may be divided between differentmodules or may be performed by the re-use of the same modules fordifferent functions.

In this embodiment, the functional modules 56-66 comprise: a 30 positiondetermination module 56 for processing images received from thestereoscopic cameras 10, a model generation module 58 for processingdata generated by the 30 position determination module 56 and convertingthe data into a 30 wire mesh model of an imaged computer surface; agenerated model store 60 for storing a 30 wire mesh model of an imagedsurface; a target model store 62 for storing a previously generated 30wire mesh model; a matching module 64 for determining rotations andtranslations required to match a generated model with a target model;and an apparatus position determination module 66.

In use, as images are obtained by the stereoscopic cameras 10, theseimages are processed by the 30 position determination module 56. Thisprocessing enables the 30 position determination module to identify 30positions of corresponding points in pairs of images on the surface of apatient 20. This is achieved by the 30 position determination module 56identifying corresponding points in pairs of images obtained by thestereoscopic camera system 10 and then determining 30 positions forthose points based on the relative positions of corresponding points inobtained pairs of images and stored data identifying the relativepositions of cameras obtaining the images.

Typically the identification of corresponding points is based onanalysis of image patches of around 16×16 pixels. In order to assistwith identifying and matching corresponding patches as has beendescribed the stereoscopic camera system 10 includes a speckle projector52 arranged to project a random or quasi random speckle pattern onto thepatient 20 being imaged so that different portions of the surface of thepatient 20 can be more easily distinguished. The size of the specklepattern is selected so that different patterns will be apparent indifferent image patches.

The position data generated by the 30 position determination module 56is then passed to the model generation module 58 which processes theposition data to generate a 30 wire mesh model of the surface of apatient 20 imaged by the stereoscopic cameras 10. In this embodiment the30 model comprises a triangulated wire mesh model where the vertices ofthe model correspond to the 30 positions determined by the 30 positiondetermination module 56. When such a model has been determined it isstored in the generated model store 60.

When a wire mesh model of the surface of a patient 20 has been stored,the matching module 64 is then invoked to determine a matchingtranslation and rotation between the generated model based on thecurrent images being obtained by the stereoscopic cameras 10 and apreviously generated model surface of the patient stored in the targetmodel store 62. The determined translation and rotation can then be sentas instructions to the mechanical couch 18 to cause the couch toposition the patient 20 in the same position relative to the treatmentapparatus 16 as they were when they were previously treated.

Subsequently, the stereoscopic cameras 10 can continue to monitor thepatient 20 and any variation in position can be identified by generatingfurther model surfaces and comparing those generated surfaces with thetarget model stored in the target model store 62. If it is determinedthat a patient has moved out of position, the treatment apparatus 16 canbe halted and the patient 20 repositioned, thereby avoiding irradiatingthe wrong parts of the patient 20.

In addition to monitoring the positioning of the patient 20, thecomputer 14 also includes an apparatus position determination module 66.As will be described, the apparatus position determination module 66 isarranged to process images captured by the stereoscopic camera system 10and identify the portions of the images corresponding to the markers30-40 attached to the treatment apparatus 16 and the mechanical couch18. The apparatus position determination module 66 is then arranged toutilize the identified portions of the images to determine thepositioning of the treatment apparatus 16 and the mechanical couch 18.This enables the positioning of treatment apparatus 16 and themechanical couch 18 to be monitored in real time simultaneously with themonitoring of the positioning of the patient 20. The positioning of thetreatment apparatus 16 and the mechanical couch 18 can then be comparedwith a predefined treatment plan to confirm that the correct portions ofthe patient are being irradiated.

Additionally determining the current position and orientation of thepatient 20, treatment apparatus 16 and the mechanical couch 18 enablesthe computer system 14 to determine if a collision is likely to occurand if necessary provide a warning or halt treatment before such acollision happens.

The processing of the apparatus position determination module 66 willnow be described with reference to FIG. 4 which is a flow diagram of theprocessing undertaken by the apparatus position determination module 66and FIGS. 5A and 5B.

In this embodiment the apparatus position determination module 66operates in parallel with the 30 position determination module S6 andthe model generation module 58 to determine the current position andorientation of the treatment apparatus 16 and mechanical couch 18 whilethe 30 position determination module 56 and the model generation module58 generates a wire mesh model of the surface of a patient 20 beingmonitored.

The apparatus position determination module 66 is invoked (s1) when itreceives a pair of images from the stereoscopic camera system 10. Theseimages will comprise images obtained from the image detectors behind thelenses 46 of the stereoscopic camera system 10 and hence will be imagesof the area of view of the stereoscopic camera system 10 obtained fromslightly different viewpoints.

Examples of images of the mechanical couch 18 and treatment apparatus 16at different points in time in different orientations are illustrated inFIGS. 5A and 5B. More specifically, FIGS. 5A and 5B illustrate views ofthe mechanical couch 18 and treatment apparatus 16 where the gantry 24of the treatment apparatus has been rotated relative to the main body 22in FIG. 5B relative to the position in FIG. 5A. In FIGS. 5A and 5B, inaddition to the treatment apparatus 16 and the mechanical couch 18, asurface 70 corresponding to a portion of a patient 20 being monitored isillustrated in broken outline.

As can be seen in FIGS. 5A and 5B in the images representations of themarkers 30-40 can be seen and the positions and locations of the markers30-40 are indicative of the location and orientation of the treatmentapparatus 16 and the mechanical couch 18. Also as will be apparentviewing FIGS. 5A and 5B specific markers are obscured from viewdepending on the location and orientation of the treatment apparatus 16and the mechanical couch 18 and also the portion of the image occupiedby the patient 20 being monitored.

Having received images from the stereoscopic camera 10, the apparatusposition determination module 66 then (s2) proceeds to identify therepresentations of the markers 30-40 in the obtained images.

In this embodiment this is achieved in a two-step process. Initially theapparatus position determination module 66 performs a thresholdingoperation on the images to identify portions of the images which areparticularly bright. These portions of the images should correspond tothe locations of the markers 30-40 as, as has been explained, themarkers are arranged to be retro-reflective and illuminated by the LEDlights 50 located around the circumference of the lenses 46 of thecamera system 10.

Having performed a thresholding operation, the candidate portions ofimage are then in this embodiment checked to see if the identifiedbright portions of the image are approximately circular and hencecorrespond to the circular or spherical markers 30-40 on the treatmentapparatus 16 and mechanical couch 18. A suitable determination can bemade by calculating the ratio of the numbers of pixels identified in anarea of an image having a brightness value above a particular threshold,in comparison with the numbers of pixels at the perimeter of that area.

Additionally, the apparatus position determination module 66 can alsocheck whether the identified bright areas of image are in a position inthe image where the markers would be expected to be. Where the apparatusposition determination module 66 is processing a sequence of images thiscan be achieved both by determining whether the identified bright areasare in the vicinity of areas previously identified as being markers inan earlier image. Further confirmation can be achieved by the apparatusposition determination module 66 modelling the expected appearance ofthe treatment apparatus 16 and mechanical couch 18 based on dataidentifying the treatment plan and checking whether the identifiedbright areas appear where expected.

Having identified the brightest areas in the image and filteredcandidates to exclude portions of the image which do not correspond inshape or the expected locations of the markers 30-40, the apparatusposition determination module 66 then (s3) proceeds to use theidentified areas to calculate the 30 positions of the centres of theimaged markers 30-40.

The 30 positions of the centres of the imaged markers 30-40 can bedetermined by determining the co-ordinates of the centre of a portion ofan image corresponding to a marker and then comparing the location ofthe marker between the pair of images taken by the two cameras of thestereoscopic camera system 10. In order to identify the centre of therepresentation of a marker, a weighted average of the co-ordinateshaving a brightness above a threshold can be determined. Theseco-ordinates can then be processed into 30 co-ordinates based on therelative locations of corresponding points in the two images and thelocation of the camera planes of the two image detectors containedwithin the stereoscopic camera system 10.

In some embodiments, improved accuracy of the determination of the 20co-ordinates of a point corresponding to a marker can be determined byutilizing the original grayscale image data for a section of an imageand determining an average weighted co-ordinate value for the section ofan image corresponding to a marker where the average weightedco-ordinate value is weighted by greyscale level. This is because ingeneral retro-reflective markers will appear as bright white in anobtained image but at the periphery pixels may appear to be grey whereonly part of a pixel corresponds to the surface of a marker 30-40.

Having processed a pair of images to determine the 30 location of imagedmarkers 30-40, the 30 locations are then (s4) utilized to determine theposition and orientation of the treatment apparatus 16 and mechanicalcouch 18. In making such a determination, the apparatus positiondetermination module 66 may determine the relative location andorientations of the treatment apparatus 16 and mechanical couch 18 basedon the relative positions of the identified markers 30-40. Thus forexample as has been described above the locations of the markers 30-33attached to the treatment apparatus 16 can be utilized to determine therotational orientation of the gantry 24 relative to the main body 22 ofthe treatment apparatus 22. Similarly, the rotational position of thecollimator 24 can be inferred from the positions and grouping of themarkers 34-38 attached to the collimator 24 etc.

Having determined the position and orientation of treatment apparatus 16and mechanical couch 18 based on the images obtained by the stereoscopiccamera 10, the apparatus position determination module 66 can then (s5)compare the measured position and orientation with the position andorientation the treatment apparatus 16 and mechanical couch 18 would beexpected to be in based on a treatment plan. More specifically theapparatus position determination module 66 can compare the expectedpositions of the treatment apparatus 16 and mechanical couch 18 relativeto both the iso-centre of the apparatus and also with the estimatedlocation of a tumour being treated based on the surface model of thesurface of the patient being treated. Thus in this way the apparatusposition determination module 66 can provide a real time measurement ofa deviation from the treatment plan and provide a warning if thedeviation is greater than a threshold amount.

Finally in addition to monitoring and detecting any deviation from atreatment plan the apparatus position determination module 66 can alsoutilize the measured orientation and location for the treatmentapparatus 16 and mechanical couch 18 and the measurement of the surfaceof a patient 20 to determine whether any portion of the treatmentapparatus 16 is dangerously close to colliding with either the patient20 or the mechanical couch 18 and provide a warning or halt treatment inthe event that any such collision is liable to occur.

Although in the embodiment described in detail above, portions of imagescaptured by a stereoscopic camera system 10 which are not utilized togenerate a model of the surface of a patient 20 is described as beingutilized to monitor the orientation and positioning of a treatmentapparatus 16 and a mechanical couch 18, it will be appreciated that inother embodiments other additional forms of monitoring could beundertaken.

In particular, when undertaking radiotherapy, particularly in the caseof brain tumours or the like, frequently patients are required to wear arigid face mask to hold their head in a fixed position during treatment.If the wrong face mask is worn, this can be very dangerous as the facemasks are formed to be specific to individual users and use of the wrongmask may not secure a patient properly leading to the possibility ofmovement and hence irradiating the wrong part of the patient 20. This isa particular problem in the case of brain tumours where damage toadjacent areas of the brain can cause very significant problems. Theapplicants have appreciated that, when utilizing a stereoscopic camerasystem 10 for patient positioning, potions of the periphery of an imagecorresponding to part of a face mask could be utilized to confirm thatthe correct face mask was being used. In such a system instead of usingsuch a section of the image for patient monitoring, the face mask couldbe marked in a distinctive way, for example with a bar code or the likeand when imaging a patient the presence of the bar code could bedetected and checked to confirm that the correct face mask was beingused.

Although in the above system a monitoring system based on the projectionof a speckle pattern onto the surface of a patient and the tracking ofmarkers 30-40 attached to a treatment apparatus 16 and mechanical couch18 has been described, it will be appreciated that other monitoringsystems could be utilized. Thus for example rather than generating amodel of the surface of a patient using the projection of a specklepattern other approaches such as the use of structured light could beused. Also rather than monitoring the location of markers, in someembodiments models of the surface of the treatment apparatus 16 andmechanical couch could be generated and used to monitor the positioningof a patient relative to the treatment apparatus. Monitoring themovement of the treatment apparatus 16 and mechanical couch 18 usingmarkers 30-40 is, however, preferable as this can be achieved byundertaking limited processing and therefore can be undertakensimultaneously with the modeling of the surface of a patient 20 inreal-time without excessive resource demands. Additionally the use ofmarkers 30-40 for determining apparatus location is more reliable thantrying to infer apparatus orientation based on generating a model of anapparatus surface as much of an apparatus will be self similar and hencethe apparatus would need to be monitored over a wide field of view inorder for its position and orientation to be established.

Although the embodiments of the invention described with reference tothe drawings comprise computer apparatus and processes performed incomputer apparatus, the invention also extends to computer programs,particularly computer programs on or in a carrier, adapted for puttingthe invention into practice. The program may be in the form of source orobject code or in any other form suitable for use in the implementationof the processes according to the invention. The carrier can be anyentity or device capable of carrying the program.

For example, the carrier may comprise a storage medium, such as a ROM,for example a CD ROM or a semiconductor ROM, or a magnetic recordingmedium, for example a floppy disc or hard disk. Further, the carrier maybe a transmissible carrier such as an electrical or optical signal whichmay be conveyed via electrical or optical cable or by radio or othermeans. When a program is embodied in a signal which may be conveyeddirectly by a cable or other device or means, the carrier may beconstituted by such cable or other device or means. Alternatively, thecarrier may be an integrated circuit in which the program is embedded,the integrated circuit being adapted for performing, or for use in theperformance of, the relevant processes.

1. A radiotherapy treatment system, comprising: a movable radiotherapytreatment apparatus configured to focus a beam of a radiation source ona treatment target of a patient placed on a movable couch, wherein themovable couch is configured for positioning said patient to align saidtreatment target with said beam of the radiation source, whereinpredefined positions of the movable couch and said movable radiotherapytreatment apparatus during a course of radiotherapy treatment isoperated according to a predefined treatment plan; wherein the systemfurther comprises: a camera system configured to obtain images of saidmovable radiotherapy treatment apparatus, said movable couch and saidpatient; and a control system, configured to: receive said images fromsaid camera system; obtain positions of the movable radiotherapytreatment apparatus and the movable couch during the course ofradiotherapy treatment; and compare said obtained positions of themovable radiotherapy treatment apparatus and said movable couch withsaid predefined positions according to said predefined treatment plan,generate from said images, a patient surface model of at least a part ofsaid patient containing said treatment target; and align said generatedpatient surface model with a computer model of said movable radiotherapytreatment apparatus and said movable couch, wherein said alignment ofsaid patient surface model is tracked in relation to said positions ofthe movable radiotherapy treatment apparatus and the movable couch. 2.The radiotherapy treatment system according to claim 1, wherein thecamera system includes a projector for projecting a pattern of lightonto at least part of said patient; and a stereoscopic camera operableto obtain a sequence of images, within a defined field of view, of saidpatient, wherein the control system is configured to process portions ofthe sequences of images, to track relative positioning of the movableradiotherapy treatment apparatus relative to the surface model of thepatient during the course of radiotherapy treatment, and compare therelative positioning of the movable radiotherapy treatment apparatusrelative to the surface model of the patient with an expected relativepositioning based on a predefined treatment plan to identify disparitiesbetween the relative positioning of the treatment apparatus relative tothe patient and the expected relative positioning based on thepredefined treatment plan.
 3. The radiotherapy treatment systemaccording to claim 1, further comprising one or more markers attached tothe movable couch or the movable radiotherapy treatment apparatus,wherein said control system is configured to track the positionings ofthe movable radiotherapy treatment apparatus and the movable couch bytracking a marker configuration identified from the received images. 4.The radiotherapy treatment system according to claim 3, wherein themarkers are provided on a surface of the movable couch and/or themovable radiotherapy treatment apparatus.
 5. The radiotherapy treatmentsystem according to claim 3, wherein the one or more markers areretro-reflective markers, and the control system is further configuredto identify portions of the received images which correspond to the oneor more markers by performing a thresholding operation.
 6. Theradiotherapy treatment system according to claim 1, wherein the moveableradiotherapy treatment apparatus comprises a gantry operable to rotateabout an axis relative to a main body of the movable radiotherapytreatment apparatus and the control system is configured to determinerelative rotation of the gantry about the axis based on the receivedimages and the computer model.
 7. The radiotherapy treatment systemaccording to claim 1, wherein the control system is further configuredto determine the relative positioning of the movable radiotherapytreatment apparatus relative to the surface model of the patient andgenerate a warning if the movable radiotherapy treatment apparatus islikely to collide with the patient or the movable couch.
 8. Theradiotherapy treatment system according to claim 1, further comprising aface mask having distinctive markings thereon, and wherein the controlsystem is configured to process image data corresponding to portions ofthe images which are not utilized to generate a patient surface model todetect presence of the distinctive markings on said face mask andprovide a warning if the markings do not correspond to expected markingsfor the patient being treated.
 9. A control system configured to:receive positioning data from a movable radiotherapy apparatus and amovable couch of a radiotherapy system; receive images from a camerasystem positioned in the radiotherapy system; generate from the images,a patient surface model of at least a part of the patient containing atreatment target; and align the generated patient surface model with acomputer model of the movable radiotherapy treatment apparatus and themovable couch, wherein the control system is configured to track thealignment of the patient surface model in relation to the trackedpositions of the movable radiotherapy treatment apparatus and themovable couch.
 10. Control system according to claim 9, furtherconfigured to output control signals for controlling the relativepositions of the mechanical couch and the treatment apparatus in alaterally, vertically, longitudinally, and rotationally direction. 11.Control system according to claim 9, wherein the control system isfurther configured to determine the relative positioning of the movableradiotherapy treatment apparatus relative to the surface model of thepatient and generate a warning if the movable radiotherapy treatmentapparatus is likely to collide with the patient or the movable couch.12. Control system according to claim 9, wherein the control system isconfigured to track the positionings of the movable radiotherapytreatment apparatus and the movable couch by tracking a markerconfiguration provided on the movable radiotherapy treatment apparatusand/or on the movable couch, where the marker configuration isidentified from the received images.
 13. Control system according toclaim 12, wherein the control system is configured to identify portionsof the received images which correspond to the marker configuration byperforming a thresholding operation.
 14. Control system according toclaim 9, wherein the control system comprises a position determinationmodule configured to receive the images from the camera system and toprocess the images to identify corresponding points in pairs of imagesobtained by the camera system and to forward the processed images to amodel generation module for generating at least the patient surfacemodel.
 15. Control system according to claim 9, wherein the controlsystem comprises a model generation module operable to process theimages from the cameras and generate the patient surface model. 16.Control system according to claim 9, wherein the control systemcomprises a generated model store unit configured to store the generatedpatient surface.
 17. Control system according to claim 9, wherein thecontrol system comprises a target model store configured for storing atarget patient surface model saved from an earlier session using theradiotherapy system, wherein the control system is configured tocalculate the difference between the generated patient surface from theimages with the target patient surface model and to output alignmentpositioning instructions to the mechanical couch based on the differencecalculated.
 18. Control system according to claim 9, wherein the controlsystem comprises a matching module configured to determining rotationsand translations required to match the generated patient surface modelwith the target patient surface model.
 19. Control system according toclaim 9, wherein the control system comprises an apparatus positiondetermination module operable to identify the couch and the treatmentapparatus in said images and to utilize the identified couch andtreatment apparatus positions to track the positions of the movableradiotherapy treatment apparatus and the movable couch of saidradiotherapy system in the images during the course of radiotherapytreatment using the received images.
 20. A radiotherapy treatmentsystem, comprising: a movable radiotherapy treatment apparatusconfigured to focus a beam of a radiation source on a treatment targetof a patient placed on a movable couch, wherein the movable couch isconfigured for positioning said patient to align said treatment targetwith said beam of the radiation source, wherein predefined positions ofthe movable couch and said movable radiotherapy treatment apparatusduring a course of radiotherapy treatment is operated according to apredefined treatment plan; wherein the system further comprises: acamera system configured to obtain images of said movable radiotherapytreatment apparatus, said movable couch and said patient; and a controlsystem, configured to: receive said images from said camera system;obtain positions of the movable radiotherapy treatment apparatus and themovable couch during the course of radiotherapy treatment; and generatefrom said images, a patient surface model of at least a part of saidpatient containing said treatment target; and align said generatedpatient surface model with a computer model of said movable radiotherapytreatment apparatus and said movable couch, wherein said alignment ofsaid patient surface model is tracked in relation to said positions ofthe movable radiotherapy treatment apparatus and the movable couch.